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Abstract. The iRead EU Project has released literacy games for Spanish, German, 
Greek, and English for L1 and L2 acquisition. In order to understand the impact of 
these games on reading skills for L1 German pupils, the authors employed an eye- 
tracking recording of pupils’ readings on a weekly basis as part of an after-school 
reading club. This work seeks to first understand how to interpret the eye-tracker 
data for such a study. Five pupils participated in the project and read short texts over 
the course of five weeks. The resulting data set was extensive enough to perform 
preliminary analysis on how to use the eye-tracking data to provide information on 
skill acquisition looking at pupils’ reading accuracy and speed. Given our set-up, 
we can show that the eye-tracker is accurate enough to measure relative reading 
speed between long and short vowels for selected 2-syllable words. As a result, eye- 
tracking data can visualize three different types of beginning readers: memorizers, 
pattern learners, and those with reading problems. 


Keywords: serious games, literacy games, elementary educational games, self- 
evaluation, pupils’ perspectives, technology appropriation. 

1. Introduction 

Adaptive literacy games provided by iRead EU Horizon Project (www.iread- 


project.eu) are deployed into elementary school classrooms. We employed an eye- 
tracker to understand how to study reading skills in order to use the eye-tracker 
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as an evaluation method in future work. The underlying thesis is that the level of 
understanding orthography should have an effect on eye movement that can be 
measured with an eye-tracker (Behrmann & Bub, 1992; Rau, Moll, Snowling, & 
Landerl, 2015). Age and reading skills are correlated (Blythe & Joseph, 2011). The 
work presented here forms a further step in understanding how eye-trackers can be 
used to visualize reading skills for the key orthographic principles of the German 
language, namely the pervasive long and short vowel grapheme patterns. This case 
is for beginning readers because they are still decoding the letters. As reading skills 
increase, changes in the movements of the eye will reflect the decrease in cognitive 
processing difficulty. We may expect to observe differences between equally old 
children who vary significantly in reading skills. Processing difficulty is reflected 
in the reader’s eye movement behavior (Blythe & Joseph, 2011). Haikid, Bertram, 
Hy6na, and Niemi (2009) found that less able readers aged between eight and ten 
have slower reading speeds and a smaller perceptual span than more able readers 
have. Based on the literature, we expect to be able to measure minor differences in 
reading speed due to vowel length in 2-syllable words, such as ‘Hase’ vs. ‘hasse’ in 
order to determine reading accuracy. 


2. Method 


Over a period of five weeks, five pupils at the age of 8-9 attended a weekly 
reading club with short reading sessions that were recorded with an eye-tracker 
at a frequency of 120 Hz on a high-performance laptop with a large monitor 
with a mobile Tobii Pro X3-120 (https://www.tobiipro.com/). The eye-tracker 
was re-calibrated for each pupil during each visit. The laptop was positioned for 
optimal lighting and the seat was boosted with pillows for correct positioning of 
the reader. 


Figure | shows the stimulus example presented to the pupils with invisible Points 
Of Interest (POD). A given short story was presented across multiple pages to the 
reader. The following adjustments were important: POIs should not appear at 
the beginning of a line, and POIs were stretched higher above the text to capture 
most of the eye movements above the words. Font size, word and line spacing 
were adjusted to the eye-tracker accuracy, and the font was chosen for beginning 
readers. Pupils read the text out loud to avoid skipping difficult parts of the text and 
to control the pupil’s reading speed. 


The focus of the experiment was the measurement of reading speed for four distinct 
categories of 2-syllable words: long vowel (‘Hase’) and short vowel words (‘hasse’) 
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and, within each of those categories, distinguishing between high frequency and 
low frequency words. This allows us to study whether the beginning readers are 
able to pronounce the words correctly by distinguishing vowel length based on 
orthographic patterns. The frequency dimension allows us to distinguish readers 
that can generalize to unknown words by applying orthographic patterns vs. those 
only memorizing high frequency words. 


Figure 1. The image shows POI marked for retrieving data from the eye-tracker 


Viele fe dor| Acs auf dem Land oder im 
Die Rentiere a ouf dem Land. 


(Baten) 
Dort sind auch 


: D> {saen) j 
Viele [hanger Votal-hsufa C] | SQ 


Word frequency was determined through the use of a dictionary provided by 
Heidelberg University (Schroeder et al., 2014) with word frequency counts for 
different age groups. Age group for 8-10 years old children was used. 


Table 1. Number of 2-syllable words used for the data analysis by week and 


pupil 
Number of Words Read 
Week | Pupil 1 Pupil 2 Pupil 3 Pupil 4 Pupil 5 
1 34 37 37 39 19 
2 54 70 74 21 58 
3 86 85 84 70 106 
4 86 81 84 29 47 


The Total Fixation Duration (TFD) measurement from the eye-tracker was used 
for all calculations. The final data set contained four school visits for five pupils, 
reading nine texts (1,432 words) over a time span of five weeks as listed in Table |. 
The data was cleaned as follows before normalizing for further analysis: 
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repetitive words were eliminated because they lead to quick memorization 
for all readers; 


first words in lines were not used; 
apparent outliers were excluded; 
zero readings were excluded; and 


misspelled words in the stimuli. 


Normalizing the data eliminated overall reading speed in order to focus on relative 
speed for vowel duration, enabling furthermore direct comparison between readers. 


3. 


Results 


After analyzing the data, we identified three reading profiles: 


Pattern recognizer: a good reader having acquired the patterns and able to 
distinguish between long and short vowels regardless of the frequency of 
the word. We can assume this reader can generalize to new words and is 
therefore a strong reader. 


Memorizer: a reader that has memorized frequent words, without 
understanding the orthographic pattern and will make reading mistakes in 
rare or new words. Memorizers sound like fluent readers on familiar texts 
like school books but may not move on easily to unknown texts. 


Problematic reader: these readers may need more time to read the short 
vowel because they did not understand the orthographic concept of the 
double consonant. By mistakenly choosing to read the short vowel as a 
long one, they run into a semantic conflict and this hesitation is visible in 
a longer time spent on reading this word. ‘Sie hassen Eis’ (They hate ice- 
cream) is read as *Sie hasen Eis (They bunny ice-cream). 


Figure 2 depicts the profile of a Pattern recognizer. The boxplots show the 
distributions of values of the collected data for that pupil, showing high frequency 
words are read faster than low frequency words and that short vowel words are read 
faster than long vowel words. 
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Figure 2. Example profile of a Pattern recognizer (y-axis denotes normalized TFD) 
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4. Conclusions and future work 


The path to the current understanding of how to record and analyze the data was not 
straightforward. This paper has shed some light on how to set up the eye-tracker 
correctly and select the optimal measurement, clean the resulting data appropriately, 
and then use it for analyzing relative reading speed for 2-syllable words. What 
remains to be done are additional data collections, detailed statistical analyses, 
and generalization to a larger number of beginning readers (Berkling & Franken, 
2019). Our findings will form the basis for measuring reading improvements over 
time. In addition, we plan to use reading comprehension questions to validate 
reading profiles. 
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